-
Notifications
You must be signed in to change notification settings - Fork 63
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
snapshots refactoring #1970
snapshots refactoring #1970
Conversation
e7edb0f
to
01c1d2a
Compare
01c1d2a
to
73ca1ad
Compare
…th_metadata optional
73ca1ad
to
cf43085
Compare
use ptrdiff_t difference_type in iterators
35c156e
to
0b449ef
Compare
0b449ef
to
d09764a
Compare
Making read_senders=true always has no tangible effect on performance.
with tx.set_sender() in decode_word_into_tx():
without tx.set_sender()/senders_data in decode_word_into_tx():
|
@canepat I was able to execute pre-downloaded snapshots from scratch to the last block I have in snapshots (18.8M - 1) locally on macOS. It took about 4 days.
|
Great work! |
Previously the Snapshot subclasses were responsible for everything:
Most of the logic was copy-pasted 3 times with minor variations.
Now the Snapshot logic is reorganized to decouple storage engine from the Ethereum domain objects.
The copy-paste is reduced.
The same design structure can be applied to support more Snapshot/Index types in the future.
The Repository data model is changed to contain a continuous sequence of full snapshot bundles (see (5) below).
Changes:
SnapshotWordDeserializer represents the snapshot word format. Multiple deserializers can be used for the same .seg (e.g. to get words as raw RLP).
Snapshot represents a .seg file read-only view and provides iteration over entries with a custom SnapshotWordDeserializer.
Example to open a snapshot:
Example to open an index:
A snapshot reader is a type-safe iterator over Snapshot words. A reader is bound with a particular SnapshotWordDeserializer. Readers don't depend on indexes. They assume that the snapshot was open for reading.
Example to iterate over headers:
Each query is a separate struct. Queries usually depend on indexes. In this case they assume that the index was open for reading.
Common queries are implemented in basic_queries.hpp: FindByIdQuery, FindByHashQuery, RangeFromIdQuery. They are then bound with a particular reader to produce results:
Example to find a header by hash:
Some queries have custom logic, for example: BodyTxsAmountQuery.